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ABSTRACT 



The arts enrich the culture and individual lives 
immeasurably, but what impact do arts -based programs have in preventing 
juvenile delinquency? To address this question, the YouthARTS Development 
Project, with the technical assistance of the Office of Juvenile Justice and 
Delinquency Prevention (OJJDP) , brought together Federal agencies, national 
art organizations, and a consortium of local arts agencies to develop and 
assess arts-based prevention programs for at-risk youth. This bulletin 
describes the evaluation and its positive findings for YouthARTS programs in 
Atlanta (Georgia) , Portland (Oregon) , and San Antonio (Texas) . The bulletin 
states that the lessons learned by the Art-as-Work (Atlanta) , Youth Arts 
Public Arts (Portland) , and Urban smARTS (San Antonio) programs will help 
other agencies to improve their arts programs, achieve project goals, and 
recognize the importance of evaluating arts-based programs for at-risk youth. 
According to the bulletin, arts-based delinquency programs have a promising 
future, and objective assessments such as these show that people are on the 
right path toward realizing that future. (BT) 
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The Youth ARTS 
Development Project 



Heather J. Clawson and Kathleen Coolbaugh 

The YouthARTS Development Project, ini- was known about their effectiveness in 
tiated in 1995, is a collaborative effort actually preventing juvenile problem be- 

among Federal agencies, national arts or- haviors. In 1996, the President’s Commit- 
ganizations, and a consortium of three tee on the Arts and the Humanities found 
local arts agencies in Atlanta, GA; Port- that “only a handful of studies have 

land, OR; and San Antonio, TX, designed begun to document the positive relation- 



to identify, implement, and refine effective 
arts-based delinquency prevention pro- 
grams. To support this purpose and, at 
the same time, to provide much-needed 
information to the broader arts com- 
munity about the efficacy of such pro- 
grams, the National Endowment for 
the Arts (NEA) and the U.S. Department 
of Justice’s Office of Juvenile Justice 
and Delinquency Prevention (OJJDP) part- 
nered in 1995 to conduct a national evalu- 
ation of the project. 

The YouthARTS Development Project was 
spearheaded by the National Assembly of 
Local Arts Agencies (NALAA), now Ameri- 
cans for the Arts, in recognition of the 
expanding role of local arts agencies in 
using the arts to address social and com- 
munity development issues. Using data 
from a 1995 survey of its 3,800 members, 
NALAA found that increasing numbers of 
local arts agencies were pursuing social 
change projects, most of which were pro- 
grams for youth with the common goal of 
reducing problem behaviors such as 
school failure, drug use, delinquency, and 
teen pregnancy (Mulcahy, 1996). 

Despite the growing interest and invest- 
ment in these programs, however, little 
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ship between program participation and 
cognitive development, motivation to 
learn, organization, self-perception, and 
resiliency” (Weitz, 1996). After a compre- 
hensive review of existing evaluation 
research on the impact of arts-based 
programs for at-risk youth, The RAND 
Corporation similarly concluded that “in- 
teresting arts programs abound, [but] 
few provide good evaluations of their out- 
comes” (McArthur and Law, 1996). Al- 
though these studies found some evi- 
dence to substantiate the hypothesis that 
arts-based programs could foster desir- 
able participant outcomes, both the 
quantity and quality of existing evalua- 
tions made it difficult to verify the pro- 
grams’ success. According to both the 
President’s Committee on the Arts and 
the Humanities and RAND, more con- 
trolled evaluations of arts programs for 
at-risk youth were needed to determine 
whether such programs are effective and, 
if so, what features of the programs are 
most important to their success. 

The YouthARTS Development Project was 
launched with three overarching goals: 
to enhance program development and 
capacity-building in local arts agencies, to 
identify effective arts-based delinquency 
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A Message From OJJDP 

The arts enrich our culture and our 
lives immeasurably, but what impact 
do arts-based programs have in pre- 
venting juvenile delinquency? Until 
recently, there has been little objec- 
tive evidence available to determine 
whether youth arts programs en- 
hance participant skills that reduce 
the risk of involvement in delinquency. 

To address this need, the YouthARTS 
Development Project brought togeth- 
er Federal agencies, national art 
organizations, and a consortium of 
local arts agencies to develop and 
assess arts-based prevention pro- 
grams for at-risk youth. 

OJJDP has provided technical assist- 
ance and funding in support of a 
national evaluation of the YouthARTS 
Development Project. This Bulletin 
describes the evaluation and its pos- 
itive findings for YouthARTS programs 
in Atlanta, GA; Portland, OR; and San 
Antonio, TX. 

The lessons learned by the Art-at- 
Work (Atlanta), Youth Arts Public Arts 
(Portland), and Urban smARTS (San 
Antonio) programs will help other 
agencies to improve their arts pro- 
grams, achieve project goals, and re- 
cognize the importance of evaluating 
arts-based programs for at-risk youth. 

Arts-based delinquency prevention 
programs have a promising future. 
Objective assessments, such as 
those featured in these pages, show 
that we are on the right path toward 
realizing that future. 



prevention and intervention programs, 
and to disseminate information about pro- 
gram planning, implementation, and evalu- 
ation nationally. To accomplish these 
goals, three local arts agencies — the Fulton 
County Arts Council in Atlanta, GA; the 
Regional Arts and Culture Council in Port- 
land, OR; and the San Antonio Department 
of Arts and Cultural Affairs in San Antonio, 
TX — with support from the National En- 
dowment for the Arts and several founda- 
tions, each collaborated with local social 
service agencies beginning in the fall of 
1996. 1 The arts agencies implemented arts- 
based demonstration programs for youth 
who were at risk of engaging in problem 
and delinquent behaviors. Figure 1 depicts 
the overall YouthARTS program model. 
Although all three of the YouthARTS 
programs — Art-at-Work in Atlanta, GA; 
Youth Arts Public Art in Portland, OR; and 
Urban smARTS in San Antonio, TX — were 
based on the same risk- and protection- 
focused approach, they served different 
target populations and provided different 
arts-based activities. 

National Evaluation 
of the YouthARTS 
Development Project 

The national evaluation of the YouthARTS 
Development Project was designed as a 
cross-site evaluation with both process 
and outcome components. The process 
component gathered information on pro- 
gram implementation and operations, and 
the outcome component assessed the 
extent to which the three YouthARTS 
programs had immediate and long-term 
positive effects on the knowledge, atti- 
tudes, and behaviors of program partici- 
pants. To both conserve limited evalua- 
tion resources and engage local program 
staff in the process, the evaluation design 
relied heavily on local staff and local data 
collectors hired by the programs to assist 
with much of the data collection effort. 

The process component of the evaluation 
used a qualitative approach, including a 
review of program documents and inter- 
views with staff, participants, and key 
stakeholders, to document program op- 
erations. The outcome component of the 
evaluation used a quasi-experimental de- 
sign, supported by qualitative data, to 
determine the extent to which: 

♦ The programs achieved the desired 
immediate effects on participants (e.g., 
increased art knowledge and improved 
program-related skills such as commu- 
nication and cooperation). 

o 



Figure 1: YouthARTS Development Project Design 



Conditions 




♦ The programs had the desired interme- 
diate effects on the attitudes and be- 
haviors that affect delinquency and 
academic performance (e.g., healthier 
attitudes about drug use, increased 
positive peer and adult associations, 
improved self-esteem). 

♦ The programs had the desired long- 
term impacts on juvenile delinquency 
and academic performance (i.e., de- 
creased court referrals and increased 
academic achievement). 

The goal of using a quasi-experimental 
design was to compare the attitudes and 
behaviors of program participants with 
those of matched comparison-group 
youth at multiple time points to attribute 
improvements in skills, attitudes, and be- 
haviors, with some degree of confidence, 
to program participation. Implementing 
a quasi-experimental design with these 
programs proved to be a significant chal- 
lenge, as described below in the evalua- 
tion site summaries and the section on 
evaluation lessons learned. 

Although some processes and instru- 
ments were tailored for each site, the 
overall evaluation used a common set of 
data collection procedures and instru- 
ments designed by the national evaluator. 
For the process evaluation, all interviews 
were conducted by the national evaluator. 
For the outcome evaluation, the national 
evaluator provided the following seven 
standardized instruments for use by the 
local data collectors: 

♦ Art knowledge survey tailored for 
each site. 

♦ Participant skills assessments to docu- 
ment the artists’ perceptions of partici- 
pants’ art and program-related skills 
(e.g., expressing anger appropriately, 
communicating effectively with adults 
and peers, cooperating with others, 
participating in sessions, working on 
tasks from start to finish). 

♦ Your Opinions Count (YOC) survey to 

measure changes in participants’ self- 
reported attitudes and behaviors (e.g., 
attitudes about school, drug use, and 
the future; self-esteem; peer pressure). 

♦ Participant focus group interview 
guide to collect supporting qualitative 
data about program operations and 
outcomes. 

♦ Probation officer/caseworker feed- 
back survey to obtain probation offic- 
ers’ and caseworkers’ perceptions of 
program outcomes. 







♦ Academic data form to gather aca- 
demic data for participants and com- 
parison youth. 

♦ Court information form to record 
basic demographic data and court 
information for participants and 
comparison youth. 

The evaluation was conducted from fall 
1996 through spring 1999. The overall 
evaluation design called for data to be 
collected before program participation 
(pretest), at the end of the program cycle 
(posttest), and for an identified period 
after the program cycle (14 months in At- 
lanta, 19 months in Portland, and 22 
months in San Antonio). 2 The national 
evaluator provided training and detailed 
data collection manuals to the local staff 
on how to use each of these data collec- 
tion instruments. The national evaluator 
also assisted the local sites in their selec- 
tion of procedures for choosing appropri- 
ate comparison groups and obtaining 
data from and about them. 

In fact, identifying comparison groups and 
relying on local data collectors posed the 
most formidable challenges for the out- 
come evaluation. Selecting appropriate 
comparison groups was difficult in each of 
the sites, primarily because of the need to 
identify youth similar to the program par- 
ticipants (because random assignment 
was not possible) and then to retain their 
involvement throughout the course of the 
study. Given the populations involved in 
the YouthARTS Development Project (i.e., 
system-involved or at-risk youth), this was 
also a time-consuming undertaking. Com- 
peting demands on staff time and atten- 
tion, turnover, and lack of data collection 
experience and/or commitment to the 
evaluation process all affected the timeli- 
ness and quality of some of the data col- 
lected. These constraints, coupled with 
the relatively few participants served in 
the Atlanta and Portland programs, re- 
sulted in findings that, although promising 
for arts-based prevention programs, 
should be considered preliminary. The 
small sample sizes and lack of true 
comparison groups limit evaluators’ abil- 
ity to generalize the findings or attribute 
positive outcomes unequivocally to the 
arts program interventions. Nonetheless, 
the findings presented below for each 
program are encouraging and support the 
underlying theory that arts-based pro- 
grams help contribute to reduced juvenile 
delinquency. In 1998, the Americans for 
the Arts published The YouthARTS Tool 
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Kit, containing information on program 
planning, staff training, evaluation, and 
costs and resources for arts-based preven- 
tion and intervention programs. 3 The Tool 
Kit includes a section on how to evaluate 
arts-based programs based on what was 
learned from the national evaluation. 

Each of the programs is described in the 
following sections, with a summary pre- 
sented in the table. These overviews de- 
scribe each program’s first year of opera- 
tion, including startup activities, program 
goals, youth served, and program activi- 
ties. They also describe the evaluation 
process and findings for each site, fol- 
lowed by important lessons learned in 
Atlanta, Portland, and San Antonio. Other 
agencies may find these lessons helpful 
as they evaluate their own arts-based 
programs for at-risk youth. 

Art-at-Work 

Program Description 

In September 1996, the Fulton County 
Arts Council entered into a collaborative 
partnership with the Fulton County Juve- 
nile Court to implement Art-at-Work in 
Atlanta, GA. This program was designed 
to provide art instruction, job training, 
and literacy education to a small group of 
first-time status offenders, ages 14 to 16, 
whose most serious juvenile offense was 
truancy. Program participants were re- 
ferred by probation officers of youth who 
were first-time truants on probation for 2 
years, a period consistent with the length 
of the Art-at-Work program. The program 
was limited to 15 youth to allow for inten- 
sive one-on-one interaction between the 
artists and the youth. Art-at-Work was 
designed to provide participants with: 

♦ Improved art and employment skills 
(e.g., goal setting, communication, 
sales/marketing). 

♦ Opportunities to use their new skills to 
produce, exhibit, and sell their own 
art. 

♦ Opportunities to display artwork and 
receive public recognition for their 
work. 

♦ Exposure to career opportunities in 
the arts. 

♦ Opportunities to develop positive 
relationships with adult role models 
and peers. 

♦ Improved or increased self-esteem, 
pride, discipline, commitment, 



Summary of Youth ARTS Programs 



Parameter 


Art-at-Work 
Atlanta, GA 


Youth Arts Public Art 
Portland, OR 


Urban smARTS 
San Antonio, TX 


Local collaborative 
partners 


Fulton County Arts Council 
and Fulton County Juvenile 
Court. 


Portland Regional Arts 
and Culture Council and 
Multnomah County Division 
of Juvenile Justice Service. 


City of San Antonio Depart- 
ment of Arts and Cultural 
Affairs, Department of Com- 
munity Initiatives, and San 
Antonio Independent School 
District. 


Target population 


Truant youth ages 14-16 
referred by current probation 
counselors. 


Adjudicated youth (except 
those adjudicated for sex 
offenses) ages 14-16 referred 
by current probation officers. 


Nonadjudicated, at-risk youth 
ages 10-12 referred by 
teachers, principals, and 
self-referrals. 


Capacity 


15 youth per program period. 


15 youth per unit per session 
(gang reduction unit, North 
unit, and Southeast unit). 


60 youth at each of 7 schools. 


Duration 


4 sessions (8-12 weeks each) 
with the same group of 
participants. 


1 session per unit (12 weeks 
each). 


1 session (16 weeks) per 
school per year. 


Frequency 


8 hours per week during 
school, 25 hours per week 
during summer. 


6 hours per week. 


9 hours per week. 


Staffing 


Program director, project 
manager, lead artist/program 
coordinator, professional 
artists, and probation 
counselors. 


Program director, project 
manager, professional artists, 
and probation officers. 


Program director, project 
manager, teacher liaison, 
professional artists, and 
caseworkers. 


Training 


Two-day artist training 
focusing on child develop- 
ment, conflict resolution, 
problem solving, and class- 
room management. 


Informal training of artists 
and probation officers focus- 
ing on program design, goals 
and objectives, background 
of participants, and rules and 
regulations. 


Five-day cross-training of 
artists and caseworkers 
focusing on working with at- 
risk youth, child management, 
curriculum development, and 
school rules and regulations. 



continued on next page 



responsibility, and attitudes about 
school and the future. 

♦ Increased prosocial behavior and re- 
duced alienation from others. 

In addition, youth were expected to show 
increased academic, vocational, and so- 
cial success and decreased truancy and 
other delinquent behaviors. 

During the planning stages, the Arts 
Council worked closely with the Fulton 
County Juvenile Court. The newly hired 
project manager attended juvenile hear- 
ings, met with judges and probation 
officers, and shadowed probation officers 
to learn more about the juvenile justice 



system. The chief judge assigned the di- 
rector of intake to serve as the key liaison 
to the Art-at-Work program. Over time, 
court officials, including the chief judge, 
supported the program by publicizing its 
achievements, visiting classes, and at- 
tending exhibits of participants’ work at 
the end of each program session. In addi- 
tion, juvenile court probation officers 
who referred youth to the program peri- 
odically conferred with the lead artist to 
ensure that youth were participating in 
the program and to help the artists ad- 
dress behavioral problems. 

Art-at-Work staff positions included a 
project manager, a program coordinator, 



artist instructors, and a local data collec- 
tor. Although Art-at-Work planned to hire 
a social worker to assist with the pro- 
gram, the position was never filled. The 
director of the Fulton County Arts Council 
served as the executive director of the 
program. Four full-time and three part- 
time artists were carefully selected by the 
project manager and program coordi- 
nator from a pool of qualified artists who 
had experience working with at-risk 
populations. 

A week before the program started, pro- 
gram staff conducted an artist training 
session. The curriculum included topics 



Summary of YouthARTS Programs — Continued 



Parameter 


Art-at-Work 
Atlanta, GA 


Youth Arts Public Art 
Portland, OR 


Urban smARTS 
San Antonio, TX 


Approach 


Arts-based afterschool and 
summer education and job- 
training program designed to 
serve one group of youth for 
2 years. 

Part of each day is spent 
learning art skills and 
producing saleable art. 
Projects have included 
furniture design and appli- 
cation, ceramics, mosaics, 
photography, drama, and 
computer graphics. Part of 
each day is spent learning 
entrepreneurship and 
planning exhibits. Students 
help organize exhibits at 
the end of each session. 

All proceeds support the 
program. 


Arts program designed to 
involve youth in the produc- 
tion and administration of a 
public arts project, from 
design to production and 
public exhibition. 

Youth work with the artists 
twice per week during after- 
school hours. Probation 
counselors are present at 
each session to help artists 
control problem behaviors. 

Each session focuses on a 
different art medium. The 
media include printmaking, 
photography, poetry, drama, 
and videography. 

A final exhibit or presentation 
designed to provide youth 
with recognition for their 
accomplishments is scheduled 
for the end of each session. 


Afterschool arts education 
program for youth at seven 
schools. 

Each school is assigned 
three artists who design and 
implement the art activities 
(i.e., dance, visual arts, drama, 
creative writing, and story- 
telling). The program also 
provides educational field 
trips. 

Transportation home from the 
program and snacks during 
program hours are provided. 

Case management is provided 
by the Youth Services Division 
of the Community Initiatives 
Department. Every youth 
referred to the program re- 
ceives a home visit from 
a caseworker for intake and 
assessment of the youth and 
family. 


Incentive 


Students receive $5 per hour 
during school year and $100 
per week during summer 
vacation. 


Participants from the North 
and Southeast units receive 
time off probation or commu- 
nity service hours for suc- 
cessful completion. 


N/A 






Gang reduction unit partici- 
pants are given a $100 incentive 
and are required to participate. 




Intended outcome 


Art skills 


Art skills 


Art skills 




Vocational/entrepreneurial 

skills 


Vocational/entrepreneurial 

skills 


Vocational/entrepreneurial 

skills 




Life skills 


Life skills 


Life skills 




Prosocial behavior 


Prosocial behavior 


Prosocial behavior 



such as child development, conflict reso- 
lution, problem solving, and classroom 
management. 

In the fall of 1996, probation officers re- 
ferred 15 youth to the program, only 10 of 
whom were still participating regularly by 
December. Youth were not required to 
attend the program, and overall atten- 
dance was lower and more sporadic than 



expected by program staff. Reasons for 
sporadic attendance included problems 
for teen mothers in finding affordable 
childcare, schedule conflicts with other 
afterschool activities, and problems with 
transportation. During the evaluation, 
analysis of the backgrounds of youth 
who regularly attended the program 
revealed that several were not first-time 



status offenders but rather had commit- 
ted more serious offenses before program 
participation. 

The Arts Council allocated space within 
its West End Performing Arts Center — a 
location easily accessible by public trans- 
portation and within walking distance of 
many participants’ homes — for program 
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Creating a clay sculpture in the Art-at-Work 
program in Atlanta, GA. 



activities. The Art-at-Work program activi- 
ties included arts instruction in various 
art disciplines, a literacy component, and 
job skills training. A 12-week art session 
was provided during the fall-winter school 
term, followed by a 12-week spring ses- 
sion, with a break between sessions that 
corresponded with winter vacation for 
students. Each of the 12-week sessions 
operated for 2 hours after school on 
Tuesdays and Thursdays and 4 hours on 
Saturdays. During the summer, an inten- 
sive 8-week session met for 5 hours per 
day, Monday through Friday. Although not 
part of the original program design, an 
additional 8-week session, operating on 
the same schedule as the 12-week ses- 
sions, was added to the program in May 
to keep youth involved during the transi- 
tion from the end of the school year until 
the start of the summer session. On alter- 
nate Saturdays throughout the program, 
visiting artists provided additional arts 
instruction or field trip opportunities for 
the participants. The field trips included 
visits to the Atlanta College of Art, the 
Nexus Press, Seven Stages and other local 
theaters, and art exhibits. The youth par- 
ticipants, who were called apprentice art- 
ists, were paid $5 per hour during the 
school year and $100 per week during the 
summer for participating in the classes 
and producing marketable art. 
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Throughout the program year, the youth 
were divided between two studios, each of 
which focused on a particular art discipline. 
Half of the participants were assigned to 
each studio, and halfway through the ses- 
sion, the two groups switched studios. 
Separating the participants into two stu- 
dios decreased the apprentice-instructor 
ratios, and switching between studios pro- 
vided youth with an opportunity to work 
with new media and different instructors. 
The art disciplines covered during the 
year were graphic design, drawing, paint- 
ing, mosaics, sculpture, artist chairs, mu- 
rals, photography, and drama. In the stu- 
dios, each of the apprentices developed 
multiple pieces of art to display for sale at 
an exhibition at the end of the cycle and at 
least one piece that he or she could take 
home. Proceeds from the sale of the art 
helped cover the program’s operating 
costs, including stipends. This process of 
producing and selling art was designed to 
teach the participants practical business 
concepts, including production goals, in- 
ventory, and marketing. 



Evaluation Activities and 
Findings 

The outcome evaluation in Atlanta was 
almost immediately hampered by 
difficulties in identifying an appropriate 
control group. A control group could not 
be formed from the 15 youth initially 
identified as program candidates because 
they all accepted the invitation to partici- 
pate. Probation officers were then asked to 
identify another group of youth who were 
similar to the program participants with 
respect to age, sex, race, and court involve- 
ment to serve as the control group. These 
youth were offered a modest stipend to 
participate. Because only three youth ac- 
cepted, evaluators decided not to include 
them in the evaluation. Probation officers 
were asked to identify another group of 
youth for whom academic and court data 
could be collected from existing school 
and court records; they identified 12 youth 
for the control group. Of the 10 core pro- 
gram participants, complete evaluation 
data were collected for 7. Thus, the partici- 
pant and control groups for whom evalua- 
tion data were available included 7 and 
12 youth, respectively. Moreover, only 



People underestimate the power of art and the power of youth to express them- 
selves. Art-at-Work gives youth a voice. 

— Art-at-Work Program Manager 

Art gives you a way to express yourself. You can get your feelings out through your 
art without hurting anyone. 

— Art-at-Work Participant 

I think we often underestimate youth. I was surprised at how kids with no art experi- 
ence could not only increase their knowledge of art concepts but apply what they 
learned in a critique of a mural. 

— Art-at-Work Artist 

I learned how to work with other people. The people skills I learned in the program 
have helped me. I am working and want to go to business school. 

—Art-at-Work Participant 

If I wasn’t in the program I probably would have gotten pregnant again and dropped 
out of school. Now I’m still in school and doing good. 

— Art-at-Work Participant 

Since I started the program, I’ve started back to school regularly. I’ve started being 
responsible. 

— A rt-a t- Work Participan t 

If you expect very little from youth, that is what you will get! We expected high- 
quality [art] products, and that is what the youth produced. They more than met our 
expectations. 

—Art-at-Work Artist 
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archival data were collected for the con- 
trol group youth; they did not complete 
the YOC survey. Participant survey and 
archival data were augmented with inter- 
view data from the art staff about the 
program participants and from the partici- 
pants themselves. 

According to the end-of-program partici- 
pant skills assessment, all seven of the 
young artists had gained the skills neces- 
sary to produce quality art, had produced 
art, and had received public recognition 
for their work. The art staff also report- 
ed remarkable improvement in the parti- 
cipants’ enthusiasm and appreciation 
for art over the course of the program. 

As depicted in figure 2, all seven youth 
showed improvement in four of the 
program-related skills assessed by the 
evaluation: art skills, cooperating with 
others, participating, and communicating 
effectively with their peers. 

Responses from the YOC participant sur- 
vey reflect improvements in attitudes to- 
ward school, attitudes about drug use, 
and the frequency with which the youth 
engaged in delinquent behaviors. Feed- 
back from the probation officers tends to 
suggest Art-at-Work had a positive impact 
on the attitudes and behaviors of youth. 
Probation officers noted that youth 
who participated in the program 




Refurbishing a chair in the Art-at-Work 
program in Atlanta, GA. 
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Figure 2: Program-Related Skills Exhibited by Art-at-Work Participants 
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demonstrated increased self-esteem and 
an increased sense of accomplishment 
and pride and showed improved relation- 
ships with their peers and family mem- 
bers. They also believed the program was 
valuable because, in addition to being a 
fun, safe, and challenging afterschool ac- 
tivity, it enhanced future opportunities for 
some of the participants by providing 
skills that will help them academically 
and vocationally. Although the limited 
academic data collected for the two 
groups preclude drawing any meaningful 
conclusions about academic achieve- 
ment, participants indicated in interviews 
that their participation in the program 
had a positive influence on their aca- 
demic performance. Two participants de- 
cided to continue their education beyond 
high school as a result of the arts pro- 
gram, with one attending business school 
and the other enrolling in art classes at 
community college. 

Despite the disappointing quality and 
quantity of evaluation data from the Art- 
at-Work program, even cautious inter- 
pretation of the information provided by 
the youth participants and program staff 



suggests the program was beneficial. 
Recognizing the limitations of this evalu- 
ation but still committed to arts-based 
prevention, Art-at-Work is continuing to 
evaluate its program. 

Plans for the Future 

After completion of the first Art-at-Work 
program period, the project manager held 
a meeting to discuss the findings from the 
evaluation with partners from the com- 
munity, including judges and probation 
officers. In addition, former program par- 
ticipants and program staff were invited 
to provide input. The focus of the meeting 
was to reassess the program design based 
on lessons learned from the evaluation. 

As a result, the group decided to make 
several changes. 

Greater attention to recruitment. 

Although the program’s original design 
called for provision of services to truant 
youth, the court data collected for the 
evaluation verified that, indeed, youth 
who participated in the Art-at-Work 
program were more involved with the 
juvenile justice system than the program 
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originally anticipated. Under the new pro- 
gram design, greater attention is given to 
recruitment and assessment of partici- 
pants with minor delinquent offenses in- 
cluding status offenses such as truancy. 
Program staff determined that the pro- 
gram is not equipped to address the more 
serious offenders who tend to have mul- 
tiple problems such as substance abuse 
and dependency issues. 

Limited duration of the program. Rather 
than expecting youth to commit to a 2- 
year period, program staff have rede- 
signed the program to operate for 1 year. 
The attrition rates suggested that a 12- 
month period was “long enough” for 
youth. The demands on youth’s time (e.g., 
afterschool activities, full-timework, fam- 
ily commitments) and changes in interest 
made a 2-year commitment unrealistic for 
most youth. 

Continued evaluation. The project man- 
ager and others involved with the Art-at- 
Work program saw clearly how important 
evaluation results were for obtaining on- 
going program support and funding and 
for improving the overall quality of the 
program. Recognizing many problems 
with the initial evaluation of Art-at-Work 
(e.g., missing data, lack of academic data, 
poor selection of comparison group), the 
project manager approached Caliber As- 
sociates to conduct a new evaluation to 
begin in late September 2000. This evalua- 
tion is in progress. Caliber is currently 
addressing the problems of the initial 
evaluation with the project manager, and 
plans are in place for conducting a rigor- 
ous evaluation using either random as- 
signment or a true matched comparison 
design. Plans call for building the evalua- 
tion into the program’s overall design to 
allow for accumulation of data on more 
youth participants over several years, 
which will permit program staff to make 
stronger statements about the signifi- 
cance of the arts program in changing 
youth attitudes and behaviors. This ap- 
proach will not compromise the ability of 
the artists to continue working with small 
numbers of youth on a one-on-one basis. 

Youth Arts Public Art 

Program Description 

In 1996, the Portland Regional Arts and 
Culture Council, in collaboration with the 
Multnomah County Division of Juvenile 
Justice Services and other local arts 



o 




organizations, initiated the Youth Arts 
Public Art program in Portland, OR, with 
funding from Percent for Public Art. 4 
Youth Arts Public Art was designed to 
serve small groups of youth ages 14 to 16 
who were on probation in the Portland 
juvenile justice system for any status or 
delinquency offense except sex offenses. 
The goal of the program was to achieve 
the following participant outcomes: 

♦ Improved art skills. 

♦ Increased awareness of art education 
and careers. 

♦ Recognition of new skills. 

♦ Positive relationships with adult role 
models and peers. 

♦ Improved self-esteem and attitude to- 
ward the future. 

♦ Improved social skills (e.g., communi- 
cation, teamwork, empathy). 

Improvement and gains in these areas 
were expected to enhance academic and 
social success and reduce involvement in 
delinquent behaviors. 

The Regional Arts and Culture Council 
worked closely with the Division of Juve- 
nile Justice Services and arts organiza- 
tions to develop Youth Arts Public Art, 
determine its goals and objectives, and 
define the roles of the collaborative part- 
ners and their staff. Staff positions in- 
cluded a project manager, probation 
counselors, artist instructors, and local 
data collectors. The director of the Port- 
land Regional Arts and Culture Council 
served as program director. The council 
was responsible for daily program 



operations and hiring and training artists 
and other program staff, and it allocated 
Percent for Public Art funds to provide 
most of the art supplies (e.g., video equip- 
ment, costumes) and pay the artist in- 
structors. The Division of Juvenile Justice 
Services provided probation counselors to 
refer youth to the program, provide case 
management, and assist the artists in con- 
ducting the workshops. In addition, the 
probation counselors often provided trans- 
portation for participants. The Division of 
Juvenile Justice Services also provided 
snacks and gift certificates as incentives 
for youth participation. Other local arts 
organizations allocated space in their fa- 
cilities for program workshops, exhibits, 
and performances and provided video 
equipment and costumes for theater 
productions. 

Artists and counselors received orienta- 
tion training before the program sessions. 
The training focused on program design, 
goals and objectives, background of par- 
ticipants, and rules and regulations. In 
addition to this training, several artists 
attended an orientation for probation 
officers that was conducted by the Divi- 
sion of Juvenile Justice Services to help 
them better understand the program 
participants and the role of the probation 
counselors. 

The Regional Arts and Culture Council 
and the Division of Juvenile Justice 
Services identified three probation units 
to participate in the program: Southeast, 
North, and the Gang Resource Interven- 
tion Team (GRIT). The Southeast and 
North units included youth from at-risk 



We need to teach our youth that all behaviors have consequences, not just the bad 
behaviors. Youth Arts Public Art provides youth with new art skills, gives them a 
chance to use those skills in developing quality art, and gives youth a chance to 
succeed and to be rewarded for their positive behavior. 

— Youth Arts Public Art Probation Officer 

1 learned what I can do with photography. I’m pretty good with the camera. I hope to 
get a job taking pictures for a newspaper or magazine. 

— Youth Arts Public Art Participant 

In the beginning, I wanted to be sent back to court. I didn’t want to have anything to 
do with this program. Now I’m very involved and consider myself a leader. 

— Youth Arts Public Art Participant 

A lot of people got to see my work. It made me feel very proud of what I had done, 
and that doesn’t happen very often. 



— Youth Arts Public Art Participant 
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Creating monoprints in the Youth Arts Public Art project in Portland, OR. 



areas in Portland who were on probation 
for status and delinquent offenses. The 
GRIT unit included youth identified as 
gang members or involved in gang activ- 
ity. Probation officers in each unit re- 
ferred 15 youth from their probation 
caseloads to the program, based on their 
term of probation (i.e., the youth had to 
be on probation for the duration of the 
program). Although 45 youth were re- 
ferred (15 from each unit), a total of 37 
actually participated in the sessions (12 
from North, 11 from Southeast, and 14 
from GRIT). Attendance was not manda- 
tory for youth from the Southeast and 
North units, but once the youth agreed to 
participate in the program, they were ex- 
pected to commit to the entire 12 weeks. 
To promote attendance, these units of- 
fered time off probation and/or commu- 
nity service hours in exchange for active 
participation. For the GRIT unit, atten- 
dance was mandatory, and a $100 gift cer- 
tificate was offered as an incentive for 
participating in the program. Despite this 
incentive, attendance by the GRIT unit 
referrals was low and sporadic. 

Youth from each unit participated in a 12- 
week session that met for 2 hours, 3 days 
a week. The program sessions focused 
on different art forms, selected by the 
project manager and the supervisor of 
each probation unit. The North unit re- 
ceived instruction in photography (e.g., 
use of equipment, style) and poetry (e.g., 
presentation, forms of poetry), the South- 
east unit studied videography (e.g., use of 
equipment, techniques, sound, editing), 
and the GRIT unit studied theater (e.g., 
set design, costumes, script writing, 
movement). In each session, the youth 
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worked with the artists and counselors in 
small groups to develop quality public art 
and to prepare for a final exhibition or 
performance. The North unit participants 
published a book of their photography 
and poetry that focused on aspects of 
their everyday lives. The Southeast unit 
developed a short informational video 
about new juvenile justice legislation af- 
fecting youth in Portland, including inter- 
views with local politicians and judges. 
The GRIT unit performed a play that de- 
picted life in a gang and that involved 
both music and dance. 

Evaluation Activities and 
Findings 

In January 1997, probation counselors for 
each of the three probation units selected 
youth for the comparison group from 
their existing caseloads. They attempted 
to identify youth who were similar to par- 
ticipants with respect to age, sex, and 
other important factors (e.g., court his- 
tory, living in high-crime areas). Although 
the counselors invited 45 youth to volun- 
teer for the study, only 22 agreed to par- 
ticipate. These youth were given a modest 
stipend for their participation. 

As in Atlanta, data were not uniformly 
available for all youth in the two groups. 
Complete preprogram and postprogram 
data from the YOC survey were available 
for only 19 of the 37 core participants and 
13 of the 22 comparison youth. Program- 
related skills assessments were available 
for 21 participants. Only nine program 
participants were available to participate 
in data collection for the followup evalua- 
tion. Participant survey and archival data 




were augmented with interview data from 
the staff artists and with data from focus 
groups of the participants themselves. 

Based on the end-of-program participant 
skills assessment, all 21 of the participants 
had gained the skills necessary to produce 
quality art, had produced art, and had re- 
ceived public recognition for their work. 
According to the program staff, the partici- 
pants took great pride in their work and 
looked forward to the public recognition 
for their achievements. In fact, for the 
GRIT unit, it was only after the artists 
threatened to cancel the final production 
because of poor rehearsal attendance that 
the group became serious about their 
work and participation. Every youth in the 
GRIT unit was present for the final public 
performance. As shown in figure 3, most of 
the youth also showed improvement in all 
program-related skills during the program 
period, particularly in their ability to co- 
operate with others. The project manager 
and artists also observed improvement in 
the participants’ ability to work as a team 
and form new friendships. 

Findings from the YOC survey were also 
promising. Although the number of pro- 
gram participants and comparison youth 
who completed the survey was relatively 
small, there were noticeable improve- 
ments in participants’ self-reported in- 
volvement in delinquent behavior during 
the program period. Additionally, a greater 
proportion of participants than compari- 
son youth showed improvement in their 
attitudes toward school, resistance to 
peer pressure, and self-efficacy. These find- 
ings were supported by focus group data 
from the nine participants interviewed at 
the 19-month followup. These youth said 
the program taught them self-respect, ways 
to get along with others and work in a team 
environment, and, especially, the impor- 
tance of taking responsibility for their ac- 
tions. In addition, they said the program 
helped them recognize their talents and — 
most important — opened their eyes to 
opportunities and career options. 

More sobering was the fact that, al- 
though most of the youth acknowledged 
that dealing and using drugs were wrong 
and illegal (as reflected in their survey 
and focus group responses), many con- 
sidered these activities a necessity — for 
money or to escape everyday life. These 
youth were very open about their daily 
“reality,” which for many involved guns, 
violence, poverty, homelessness, and 
fear. For these youth, involvement in the 
program was very important because, as 
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Figure 3: Program-Related Skills Exhibited by Youth Arts 
Public Art Participants 
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one youth explained, “It gave us a safe 
place to spend time and it provided us 
with an opportunity to feel good about 
ourselves.” 

Because of difficulty in locating academic 
data for participant and comparison 
youth, especially those who did not regu- 
larly attend school or who attended alter- 
native schools, little can be said about 
the impact of the program on academic 
achievement. Anecdotal information from 
participants suggests, however, that the 
program was successful in changing how 
some of the youth felt about education. 

As a result of their program participation, 
some youth reported that they are getting 
their priorities straight and trying to do 
better in school. 

Although limited by the quality and quan- 
tity of evaluation data, findings for the 
Youth Arts Public Art program suggest 
the program may benefit some youth 
involved in the juvenile justice system. 
Using what has been learned about pro- 
gram implementation and impact, program 
staff are refining the program to empha- 
size what works and change what does 
not work. Additionally, reassessing which 



outcomes are realistic from a 12-week 
arts-based program with high-risk youth 
has become a priority for program staff 
during the redevelopment phase. They 
acknowledge that the problems facing 
many of the youth involved in the pro- 
gram (e.g., neglect, physical abuse, home- 
lessness) are more serious and require 
greater intervention than Youth Arts Pub- 
lic Art can offer. Staff are also reevaluating 
the appropriate target population to be 
served by the program. 

Plans for the Future 

The Youth Arts Public Art program used 
the evaluation results and lessons learned 
from the first year of operation to make 
important changes to the program. 

Greater awareness of implications of art 
media being selected. The project man- 
ager and staff learned that they need to 
be more careful when selecting the art 
media to use with different populations 
of youth. For example, they discovered 
that the use of theater with youth in the 
gang unit, who are guarded and reluctant 
to put themselves in vulnerable situations 
on stage or off, was probably a poor 



match. A less threatening art form, such 
as the videography project, might have 
been better suited for this group of high- 
risk youth. 

More emphasis on recruitment and at- 
tendance. Program staff determined that 
attendance rates would need to improve 
for the program to be successful. By serv- 
ing less serious offenders and targeting 
youth who express an interest in the arts, 
staff hope to have greater success at 
maintaining participants’ interest and in- 
creasing involvement in the program. 

Greater collaboration across sessions. 

The project manager is working on the 
development of increased collaboration 
among the various youth groups involved 
in the different sessions, which would ex- 
pose youth to more art forms without sig- 
nificantly increasing program cost. Addi- 
tionally, the joint project would provide 
opportunities for youth from diverse 
backgrounds to work together toward a 
common goal. 

Urban smARTS 

Program Description 

The City of San Antonio Department of 
Arts and Cultural Affairs initiated the Ur- 
ban smARTS afterschool program in 1993 
to prevent high-risk students, ages 10 to 
12, in San Antonio, TX, neighborhoods 
from engaging in delinquent behaviors. 
The Department of Arts and Cultural Af- 
fairs joined the YouthARTS Development 
Project in 1995 and began operating Ur- 
ban smARTS as a YouthARTS program in 
1996. Teachers and counselors at each 
participating school from the San Antonio 
Independent School District referred to 
the program primarily sixth grade stu- 
dents who were experiencing academic 
failure, had poor school attendance, and 
engaged in antisocial behavior. Urban 
smARTS provided afterschool art work- 
shops that were designed to achieve: 

♦ Improved social skills (e.g., com- 
munication with peers and adults, 
teamwork). 

♦ Improved academic performance and 
commitment to school. 

♦ Improved art skills. 

♦ Recognition for new skills. 

♦ Positive relationships with adult role 
models and peers. 

♦ Improved self-esteem and attitude 
toward the future. 
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Artwork from the Urban smARTS program in 
San Antonio, TX. 



These outcomes were expected to lead 
to academic and social success and de- 
creased delinquent behaviors. 

Since the inception of Urban smARTS in 
1993, the City of San Antonio Department 
of Arts and Cultural Affairs, the Depart- 
ment of Community Initiatives, and the 
San Antonio Independent School District 
have worked together to develop the Ur- 
ban smARTS program curriculum and de- 
fine the roles and responsibilities of the 
partnering agencies and program staff. 
Staff positions included a project man- 
ager, teacher liaison, caseworkers, and 
artist instructors. The director of the San 
Antonio Department of Arts and Cultural 
Affairs served as program director, and 
the department was responsible for hiring 
and training staff. The Department of 
Community Initiatives provided casework- 
ers for case management and for facilita- 
tion of prevention-oriented discussions 
during the program. The school district 
referred students from seven schools to 
the program and provided teacher liaisons 
to help the artists conduct the workshops, 
space in the schools for the art work- 
shops and performances, transportation 
for participants who lived farther than 
walking distance from the school, and 
snacks during the afterschool workshops. 



The program director assigned a team 
consisting of three artists (including one 
lead artist), four caseworkers, and one 
teacher liaison to each of the seven 
schools. The team was responsible for 
making decisions about program format, 
discipline, and schedules. 

Before the start of the Urban smARTS pro- 
gram in 1996, a 5-day training session was 
offered to all artists and caseworkers. 

This training focused on issues related to 
working with at-risk youth (e.g., behavior 
management), curriculum development, 
and school rules and regulations. It also 
provided an opportunity for the artists 
and caseworkers to learn more about 
each other’s disciplines and to develop 
strategies to integrate the art and 
prevention-focused curriculums. 

Although more than 400 youth were re- 
ferred to the program in the fall of 1996 
(approximately 60 youth per school), only 
1 12 participated regularly. Youth dropped 
out of the program because of lack of inter- 
est, family obligations, involvement in 
other programs, and/or problems in 
school. Although disappointing, the attri- 
tion of students from the program resulted 
in more one-on-one interaction between 
staff and the youth who did participate, 



which proved helpful for the artists, case- 
workers, and participants. Although some- 
what inconsistent in the beginning, atten- 
dance and involvement in the workshops 
increased as the 112 youth became more 
involved in the activities and bonded with 
the artists and other participants. 

Urban smARTS conducted 3-hour after- 
school art workshops on Tuesdays, 
Wednesdays, and Thursdays for 16 weeks 
at each of the seven schools. The primary 
art forms offered at each school were 
drama, dance, and visual arts. During the 
first few weeks of the program, the partici- 
pants spent part of each 3-hour workshop 
with each of the artists to learn about the 
different art forms. After several weeks, 
the youth were divided into groups based 
on their level of interest in these art forms. 
The youth then worked with the artist to 
develop skills and produce their own art. 
All of the youth also helped prepare the 
final public performance and/or exhibit 
that concluded the 16-week session. Dur- 
ing 13 of the program’s 16 weeks, case- 
workers provided weekly 30-minute ses- 
sions in a group setting in which they 
discussed prevention-oriented topics 
such as decisionmaking, conflict resolu- 
tion, self-respect, and substance abuse. 



Children need to experience a sense of accomplishment and be recognized for their 
efforts. This program enables kids to succeed at something and provides them 
positive feedback. 

— Urban smARTS Artist 

Being in the program has made me a better writer. I was never interested in writing 
before this program. Now I love to write. 

— Urban smARTS Participant 

I had never performed in front of a crowd before. It made me feel good and proud of 
what I had done. 

— Urban smARTS Participant 

Before I started the program, I was hanging out with the wrong crowd and getting 
into trouble. Now I do better in school and see myself as an artist. 

— Urban smARTS Participant 

You hear people say art saves lives. That may sound funny, but programs like Urban 
smARTS keep youth off the streets and away from dangerous situations, like gang 
activities. Our youth feel safe in our program and have positive outlooks on the future. 

— Urban smARTS Executive Director 

In October 1998, the Urban smARTS program joined nine other youth arts programs 
at the White House in being recognized with the “Coming Up Taller Award,” an award 
given to outstanding arts and humanities programs for young people. 




Evaluation Activities and 
Findings 

Conducting the evaluation of the Urban 
smARTS program proved to be a signifi- 
cant challenge, complicated by logistical, 
legal, program management, and method- 
ological issues. Of the seven participating 
Urban smARTS schools, only five partici- 
pated in the evaluation (one lacked the 
case management component, and the 
other was excluded because it was an el- 
ementary school). Project managers ex- 
cused two of these five schools from hav- 
ing to identify a control group, believing 
that, as new Urban smARTS programs, 
they should not be burdened with this 
requirement. In the three schools where 
control groups were identified, teacher 
liaisons selected sixth grade classes with 
teachers willing to administer the YOC 
survey and participate in subsequent 
data collection activities. Recognizing 
that these comparison youth were neither 
randomly assigned nor matched on key 
characteristics, project managers none- 
theless believed the selected youth would 
be comparable to the program partici- 
pants because most of the youth at each 
school shared similar demographic char- 
acteristics and faced similar risk factors. 
This proved to be true. The participant 
and comparison youth were similar on 
key factors. Ultimately, only one of the 
five schools participated in the followup 
evaluation, which included only program 
participants (i.e., no comparison group). 

For various reasons, evaluation data for 
Urban smARTS were somewhat limited. 
Late and uneven administration of some 
data collection activities, combined with 
misplacement of many postprogram sur- 
veys by the project manager, circum- 
scribed the analyses that could be con- 
ducted. Although complete preprogram 
and postprogram YOC survey data were 
available for all 112 regular program par- 
ticipants, data were provided for only 
29 comparison youth. Art knowledge, 
academic, and skills assessment data 
were available for only 93, 86, and 78 par- 
ticipants, respectively. Followup data 
were available for only 22 participants 
from one school. Survey and archival 
data were augmented with interview data 
from the artist staff and with focus group 
data from program participants. 

Participants’ gains in art knowledge were 
difficult to determine because of late 
administration of the preprogram art 
knowledge survey. Testimonies from par- 
ticipants and art instructors suggest, 



however, that the youth did learn a consid- 
erable amount about the different art forms 
that were taught, and, more important, 
youth had a greater interest in and appre- 
ciation for the arts fitter being involved in 
the program. The desire expressed by 
youth for the Urban smARTS program to be 
offered in sixth grade and ninth grade, both 
transitional years for most students, was 
evidence of their enthusiasm for the 
program. 

Findings from the end-of-program partici- 
pant skills assessment show that most 
participants demonstrated program- 
related skills throughout the program, 
and youth who did not demonstrate the 
necessary program skills at the start of 
the program mastered these skills by the 
end. As shown in figure 4, the greatest 
changes were reported in two skill areas, 
working on tasks from start to finish and 
demonstrating art skills necessary to pro- 
duce quality artwork (i.e., skills identified 
by artists as essential, given the art form 
being taught). Many artists reported 
that most participants showed marked 
improvement in their overall social and 
art skills. 



The results of the YOC survey support the 
effectiveness of the Urban smARTS pro- 
gram as a prevention initiative. Most par- 
ticipant and comparison youth report- 
ed having favorable attitudes and engaging 
in positive behaviors at program startup. 
During the program period, however, more 
participants than comparison youth 
showed improvement in their attitudes 
about school and in their behavior (i.e., less 
self-reported involvement in delinquent 
behavior). Additionally, participants were 
more likely than comparison youth to sus- 
tain their appropriate attitudes about drug 
use, resistance to peer pressure, and sense 
of self-efficacy. The fact that program par- 
ticipants could maintain their high scores 
over the course of the program better than 
comparison group members suggests that 
the program may help buffer these youth 
against many of the risk factors that can 
affect them at this age. Official court re- 
cords also supported this finding. A re- 
view of the records found no court involve- 
ment before or during the program period 
for any of the 112 participants. Only two 
participants had committed an offense 
during the 22-month followup period (both 
cases of shoplifting). 



Figure 4: Program-Related Skills Exhibited by Urban smARTS 
Participants 
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Based on the survey data available from 
participants at one school at the 22- 
month evaluation followup, the youth 
maintained their appropriate attitudes 
about drug use, positive peer associa- 
tions, resistance to peer pressure, fre- 
quency of delinquent behaviors, and 
positive relationships with adults after 
completing the program. These findings 
were echoed during focus groups with the 
22 followup participants. Most of these 
youth reported that they felt more ma- 
ture, worked harder at school, and tried 
to be better people as a result of being 
involved in Urban smARTS. 

During these focus group discussions, 
students reiterated their enthusiasm and 
support for the Urban smARTS program. 
They enjoyed their participation and the 
opportunity to learn about the arts. In 
addition, they said the program taught 
them how to appreciate things around 
them, get along better with others, deal 
with emotions in a constructive way, and 
communicate through music and the arts. 
Several participants indicated the pro- 
gram had taught them to be more self- 
confident and helped them believe in 
themselves. Several students said that 
they “hung out with the wrong crowd” 
before participating in the program and 
that the program had helped them to stay 
out of trouble by keeping them off the 
streets and giving them something to do 
after school. 

These findings suggest that the Urban 
smARTS program was successful in meet- 
ing its goal of keeping the youth partici- 
pants engaged in positive afterschool ac- 
tivities and preventing their involvement in 
delinquent behaviors. Program casework- 
ers concurred, noting that during the stu- 
dents’ participation in the program, their 
behavior and attitudes improved and they 
became more respectful of others. 

Analysis of the academic data revealed 
less promising findings. Average grades for 
participants did not change during or after 
the program. This was not surprising to 
most of the program staff, however, be- 
cause the Urban smARTS program did not 
focus on core academic subjects such as 
math, science, and English. Some evidence 
suggests that program participation may 
have helped improve school attendance. 
Program staff are reconsidering the rela- 
tionship between participation in Urban 
smARTS and academic achievement and 
hope to refine the intended academic 
outcomes of the program in the future. 



Plans for the Future 

After completing the transition of the 
Urban smARTS program administration 
from the San Antonio Department of Arts 
and Cultural Affairs to the Department of 
Community Initiatives and hiring a perma- 
nent program director, program staff ex- 
amined the evaluation results and fo- 
cused their attention on improving 
program implementation. 

Expansion of the program in schools. 

Based on the positive findings from the 
evaluation and increased local interest in 
the program, the Department of Commu- 
nity Initiatives has expanded the program 
into other local school districts and is 
considering expanding into the San Anto- 
nio high school system. Teachers, princi- 
pals, and former participants have ex- 
pressed interest and identified a need to 
expand the program into high school to 
provide services during ninth grade. 

Redistribution of resources. Expansion of 
the program into more schools has al- 
ready resulted in the need to redistribute 
resources. For example, to extend the 
reach of the program to more schools, 
one artist works in each school rather 
than the multiple artists who worked in 
each school during the evaluation period. 
Although the implications of this change 
are unknown, it was necessary given the 
current availability of resources. 

Greater collaboration among key agen- 
cies and organizations. The program di- 
rector is focusing on collaboration among 
the caseworkers, artists, and school 
personnel (i.e., teachers, principal, coun- 
selors). Improving the working relation- 
ships among the parties is expected to 
improve program recruitment, retention, 
and operation. With greater attendance 
and better coordination of services, 
youth are more likely to experience 
positive change. 

Lessons Learned 

Do arts-based programs prevent or re- 
duce delinquent behavior among youth? 
This key question remains only partially 
answered by this evaluation. Although it 
produced considerable evidence to sup- 
port the hypothesis that such programs 
can contribute to the avoidance or 
reduction of delinquent behavior, the 
evaluation suffered several problems that 
limited the ability to definitively answer 
the question. However, the lessons 
learned from this evaluation — about both 



the programs themselves and the process 
of evaluating them — can be immensely 
valuable to the design, implementation, 
and evaluation of other youth-focused 
arts-based efforts. 

Evaluation Lessons 

Evaluating any social service program is 
challenging, especially when the program 
deals with populations who are at risk of 
becoming delinquent or already in trou- 
ble. Evaluating programs that deal with 
juvenile delinquency poses special chal- 
lenges because they serve adolescents, a 
population that requires special attention 
to data collection protocols, such as ob- 
taining parental permission for their chil- 
dren to participate in interviews or sur- 
veys and gaining the cooperation of 
system-wary youth. Moreover, these pro- 
grams are often modestly funded, with 
staff stretched in many directions, work- 
ing with too many youth in too little time. 
In order for an evaluation component to 
succeed, there needs to be careful atten- 
tion to the design, timing, and resources. 
Following are the lessons learned regard- 
ing several interrelated issues — control 
groups, sample size, data collection, and 
program stability. 

Control groups. Answering questions 
about the effectiveness of delinquency 
prevention and intervention programs 
requires the most rigorous evaluation 
design possible. An experimental design 
with random assignment of youth into 
participant and control groups, the design 
that best allows for testing whether the 
program “causes” changes in partici- 
pants, was originally planned for this 
evaluation. Because of limited recruit- 
ment (i.e., not enough youth to use ran- 
dom assignment) and concerns among 
some of the program partners about the 
ethics of not providing services to all eli- 
gible youth, a quasi-experimental design 
with matched comparison groups was 
adopted. To enable evaluators to at- 
tribute changes in participants’ attitudes 
and behaviors to the program, the com- 
parison group had to be similar to the 
group of participant youth from the start 
of the evaluation in several key factors 
(e.g., age, race, grade level, history of 
delinquency, attitudes, and behaviors). 
Although attempts were made to match 
youth on key characteristics in Atlanta 
and Portland, the participant and control 
groups had some important differences 
(e.g., history of delinquency). These 
differences and the small sample sizes 
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made it difficult to attribute with cer- 
tainty a direct relationship between pro- 
gram participation and the observed out- 
comes. Implementing rigorous evaluation 
designs requires considerable planning 
and, equally important, commitment of 
time and resources to ensure that the in- 
tegrity of the design is maintained 
throughout the course of the evaluation. 

Sample size. To some extent, the small 
sample sizes for the evaluation of the 
three YouthARTS programs were due to 
program design and were thus unavoid- 
able. An important feature of many arts 
programs, like Art-at-Work in Atlanta and 
Youth Arts Public Art in Portland, is the 
small youth-to-artist ratio. Although this 
allows for quality one-on-one instruction 
of the participants, it creates a challenge 
for the evaluator. Small sample sizes pre- 
clude testing for statistical significance 
(determining whether observed changes 
occur by chance or because of the inter- 
vention). Additionally, small sample sizes 
make it difficult to generalize the findings 
to the broader population of at-risk youth. 
Although an evaluator cannot control the 
size of a program, steps can be taken to 
avoid small sample sizes resulting from 
missing data. Because data were not uni- 
formly collected for all youth at all data 
collection points, many youth had to be 
excluded from the analyses. Every effort 
must be made to obtain complete data 
for all youth involved in the program and 
comparison groups. 

Data collection. One of the most critical 
aspects of any evaluation is data collec- 
tion. Even the most rigorous design will 
not provide meaningful results without 
the necessary data. Every evaluation 
faces resource-driven tradeoffs, however, 
and data collection is often an area where 
compromises are made. The YouthARTS 
evaluation used program staff, supported 
by local data collectors hired by each 
site, to collect the YouthARTS evaluation 
data. 

Even with detailed training and how-to 
manuals provided to assist the local staff 
with their data collection tasks, the data 
collection effort suffered. Program staff 
were very busy, focused primarily on pro- 
gram operations and service delivery. 
Many of the local data collectors were 
inexperienced and, despite guidance and 
manuals, were often unable to manage 
the challenges posed in collecting the 
necessary academic and court data. Fre- 



quent technical assistance throughout the 
data collection process could help avoid 
such data collection problems in similar 
situations in the future. 

Program stability. Outcome evaluations 
seek to answer the question, Did this pro- 
gram work? Implicit in answering the 
question is an understanding of what 
“this program” is. If the program itself is 
young and still undergoing developmental 
changes (as the YouthARTS programs 
were, especially in Atlanta and Portland), 
it is difficult to determine which program 
worked or did not work. Moreover, young 
programs face enormous challenges with 
hiring and training staff, program logis- 
tics, recruitment, and day-to-day opera- 
tions. These evolving programs resulted 
in two key problems for the evaluation: 

♦ First, it was difficult for project manag- 
ers to oversee the evaluation, ensure 
compliance with data collection re- 
quirements, and deal with the day-to- 
day demands of program operations. 

♦ Second, because of the first problem, 
both identifying the intervention itself 
and ensuring that the data collection 
instruments were measuring the appro- 
priate outcomes — those that reason- 
ably could be expected to occur as a 
result of the intervention — were 
difficult. 

Future evaluations of arts-based pro- 
grams should be done on those programs 
that have reached a level of stability that 
ensures the intervention and its desired 
outcomes are well defined. Also, if pro- 
gram staff are to be responsible for data 
collection, they must have the capacity 
and training. 

Program Lessons 

Many important lessons were learned 
about planning and implementing arts- 
based juvenile delinquency prevention/ 
intervention programs. The experiences 
of these three YouthARTS programs pro- 
vide useful insights for future program 
operation and evaluation. Information 
from both the process and outcome 
evaluation components helped program 
staff identify common factors that led to 
successful program implementation. 

Collaboration. All of the programs incor- 
porated some form of collaborative pro- 
cess. Participants noted that having a 
voice in establishing program rules, 
regulations, and workshop activities and 



having frequent opportunities to produce 
and exhibit their art were factors that 
helped them engage in the program. Pro- 
gram staff recognized the need for im- 
proved collaboration among program part- 
ners (e.g., school district, juvenile justice 
agency, art agency) and the importance of 
identifying the roles and responsibilities of 
each partner. Lack of “buy in” and under- 
standing of the program and the evalua- 
tion from the beginning resulted in unex- 
pected complications (e.g., lack of access 
to data, difficulty accessing facilities, over- 
extension of staff). Additionally, program 
staff believed that the partnership needed 
to be broadened to include other youth- 
serving agencies. They quickly learned 
that many of the youth they were serving 
had problems beyond the scope of the arts 
program and that any impact the program 
may have had on improving behavior was 
reduced by other circumstances that af- 
fected the lives of the participants. Widen- 
ing the circle of collaborative partners 
would have improved the program’s ability 
to refer youth and their families to other 
services in the community. 

Skilled, qualified artists. Artists who had 
both experience in the arts and the ability 
to work with at-risk youth tended to excel 
at engaging youth in art activities, adapt- 
ing activities to meet participants’ specific 
needs, handling problem behaviors, bond- 
ing with participants, and establishing mu- 
tual respect with the youth. YouthARTS 
participants gravitated to those artists 
who exhibited expertise in their field and 
communicated easily and respectfully 
with them. 

Onsite caseworkers and probation 
officers or counselors. Onsite casework- 
ers and probation officers or counselors 
were invaluable assets to the programs. 
These individuals served as positive role 
models for the participants, and by col- 
laborating closely with the artists, they 
helped them provide individualized art 
instruction to smaller groups of partici- 
pants. Involvement of these key players 
also provided youth with an opportunity 
to develop positive relationships with 
their probation officers or counselors. 
Participants were more involved in the 
instructional activities when the proba- 
tion officers tried to learn the material 
with them. The youth seemed more likely 
to take risks (e.g., reveal personal infor- 
mation, try new things that might embar- 
rass them) when they saw their probation 
officers taking the same risks. Finally, 
close involvement of the caseworkers and 
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probation officers or counselors allowed 
them to handle behavioral problems dur- 
ing the workshops, provide referrals, and 
work with the youth and their families to 
ensure that the participants received any 
additional support needed to develop or 
maintain positive attitudes and behaviors. 

Comprehensive training for all program 
staff. Comprehensive training covering 
topics such as at-risk youth, risk factors 
and problem behaviors, classroom man- 
agement, conflict resolution education 
and collaborations with other youth- 
serving agencies was essential for all pro- 
gram staff. The training not only served to 
orient staff to the program but also pro- 
vided them with an opportunity to discuss 
expectations, roles, and responsibilities. 
These discussions ensured that program 
staff understood the importance of col- 
laboration to achieve common goals and 
that everyone shared realistic, appropriate 
expectations about what the program 
could accomplish. 

Range of arts programs and services. Pro- 
gram staff and youth participants from all 
three programs recognized the need for 
expanded arts services in the community. 
Many of the youth expressed a continued 
interest in the arts and identified the de- 
sire to be connected with other services in 
the community that would allow them to 
pursue their art interests (e.g., additional 
art classes, jobs, internships, scholar- 
ships). Getting connected to the com- 
munity was important for these youth, 
and, although each of the programs had 
made some progress in connecting youth 
to other services, most youth felt nothing 
was available for them once the program 
ended. Program staff in all three cities are 
working to identify ways to keep youth 
connected to the arts and to their com- 
munities in the future. 

Transportation for participants. One of 

the most common reasons for poor 
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program attendance cited by participants 
and staff was lack of transportation. 

Youth attendance and participation is es- 
sential to a program’s success. Providing 
reliable transportation (e.g., school buses, 
city transportation, volunteer-driven 
vehicles) makes higher attendance rates 
possible. 

Conclusion 

The future of arts-based delinquency pre- 
vention and intervention programs is 
promising. Through continued investment 
in rigorous long-term evaluations, the 
strengths of YouthARTS and other arts- 
based programs can be identified, weak- 
nesses can be corrected, services can be 
adjusted to meet the growing needs of 
youth and communities, and effective 
models can be replicated across the Na- 
tion. Just as the lessons learned from the 
national evaluation have helped the three 
YouthARTS programs redesign and imple- 
ment their programs, the information can 
benefit others in the field who are strug- 
gling with new ways to combat the prob- 
lem of juvenile delinquency. The national 
evaluation of the YouthARTS Develop- 
ment Project has shown that providing 
youth with new skills, giving them the 
opportunities to use these skills, and of- 
fering them positive feedback and recog- 
nition for their hard work can potentially 
lead to healthier attitudes and positive 
behaviors. 

Endnotes 

1. Although the YouthARTS Development 
Project was initiated in 1995, programs 
did not begin operating until fall 1996. 

2. The followup period varied by site as a 
result of differences in program length, 
availability of followup data, and other 
administrative issues (e.g., changes in 
program management) that affected the 
timing of the followup data collection. 




3. For more information on The YouthARTS 
Tool Kit , contact Americans for the Arts at 
800-321-4510 or visit their Web site at 
www.artsusa.org. 

4. Portland’s Percent for Public Art legis- 
lation requires that 1 percent of the value 
of all public construction projects be set 
aside for the installation of public art. 

For Further 
Information 

For information about the YouthARTS De- 
velopment Project Evaluation Report , con- 
tact the Juvenile Justice Clearinghouse: 

Juvenile Justice Clearinghouse 

P.0. Box 6000 

Rockville, MD 20849-6000 

800-638-8736 

301-519-5212 (fax) 

www.puborder.ncjrs.org (Internet) 
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